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ABSTRACT 

This study investigated the problem representations 
formed by college students while solving mathematics problems. 
Problem representation characteristics indicative of understanding 
were identified by analyzing audio-tapes and written work of sixteen 
subjects, ages 16 to 24, who solved mathematics problems using the 
think-aloud technique. These characteristics fall into three broad 
categories: 1) content, 2) external code, and 3) processes involved 
in establishing the representation. This characterization is 
summarized in a problem representation instrument which can be used 
to assess the degree of understanding exhibited during problem 
solving. Significant positive correlations of the characteristics 
with follow-up tasks assumed to be indicative of understanding were 
obtained. (Author) 
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abstract 

This study investigated the problem representations formed by college 
students while solving mathematics problems. Problem representation 
characteristics indicative of understanding were identified by analyzing 
audio- tapes and written work of sixteen subjects, ages 16 to 24, who solved 
mathematics problems using the think-aloud technique. These characteristics 
fall into three broad categories 1) content, 2) external code, and 3) 
processes involved in establishing the representation. This 
characterization is summarized in a problem representation instrument whic\ 
can be used to assess the degree of understanding exhibited during problem 
solving. Significant positive correlations of the characteristics with 
follow-up tasks assumed tr be indicative of understanding were obtained. 
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Characteristics Of Problem Representation 
Indicative Of Understanding In Mathematics Problem Solving 

The study of problem solving is receiving much attention currently in 
several different areas of research (Chi, Glaser, & Rees, 1981; Greeno, 
1980; lester, 1980; Suydem, 1980). Information processing researchers have 
focused on the development of computer programs to model human problem 
solving, mathematics education researchers have focused on the processes 
used by students as they solve mathematics problems, and some psychologists 
have focused on differences between expert and novice performance on physics 
problems . 

Research in mathematics problem solving makes frequent reference to 
"understanding" a problem without using a consistent and explicit definition 
of "understanding". Even a cursory glance at the literature shows that 
"understanding" is used in a wide variety of ways. For example, Polya 
(1957) used understanding to refer to the first of four stages of problem 
rolving. The understanding stage involves identifying the unknown, the data 
and the conditions. Schoenfeld (1980) broadened Polya 's view of 
understanding by including problem analysis, design and exploration as part 
of the understanding of a problem. Kilpatrick (1968) and Days (1978) each 
defined a category of processes as understanding processes. Kilpatrick 
included such processes as identifies the unknown and draws a diagram. Days 
had a narrower view of understanding processes. He introduced a category of 
processes called representational which is distinct from the understanding 
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category. Days included "draws a diagram" in the representational category 
rather than the understanding category. In other instances researchers have 
used the term "understanding" in an intuitive sense. Lucas (1972) awarded 
points to problem solvers on the basis of whether or not they displayed an 
"understanding" of the problem. Webb (1979b) awarded points to solvers on 
the basis of whether or not they "understood what the problem was asking". 

While the term "understanding" has been used by many mathematics 
education researchers few have had the study of understanding in mathematics 
problem solving as a primary focus. It seems that such a study is both 
timely and important. In most teaching situations the teacher aims to have 
students develop an understanding of concepts and procedures. And in 
problem solving situations it is a truism that it is more desirable for a 
student to solve a problem with understanding than without. Further, most 
teachers would claim to be able to identify problem solutions which etfubit 
greater understanding and those which exhibit less understanding. In order 
to study understanding in mathematics problem solving it is necessary to 
first clarify what is meant by "understanding" iv. problem solving and to 
establish a means for assessing the degree to which it is present in 
particular solutions. 

Information processing research en problem solving is helpful at this 
point. As researchers in information processing have attempted to model 
human problem solving performance, differing aspects of problem solving have 
been identified. In particular, the process of und 3rstanding a problem has 
been distinguished from the process of solving the problem. "Understanding" 
has been defined in terms of problem representation and it is generally 
agree . that the degree of understanding ejdiibited is reflected in the rv.ture 
of the problem representation (Greeno, 1977). Consequently, seems as 
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though the study of problem representation may provide greater clarity to 
the term. Furthermore, cons t rue tivists assume that problem solvers build 
idiosyncratic knowledge structures and thus view understanding in terms of 
the mental representations constructed by the learner (von Glasersfeld. 
Stef fe, Richards, & Thompson, 1983). For the oonstructivist, understanding 
is a function of a problem solver's prior experience which influences the 
nat uire of the problem representation developed. 

Definitions 

This section presents the definitions of problem , understanding , and 
problem representation as they are used in this study. Each is defined and 
discussed. 

The definition of problem which is used in this study is: M A problem is 
a situation in which an individual or group is called upon to perform a task 
for which there is no readily accessible algorithm which determines 
completely the method of solution" (Lester, 1978, p. 54). This definition is 
consistent with the definition used by researchers in both information 
processing (Newell & Simon, 1972) and in mathematics education (Lester, 
1980). 

The definition of understanding a problem that is used in this study 
is: Understanding a problem is the building of a problem representation. 
According to this definition, understanding is a process. This definition 
is used by Simon (Simon & Hayes, 1976) and by Greeno (1977) and is 
consistent with the concept of understanding used in the literature on 
language comprehension. In the study of language the concept of 
understanding is used to refer to the construction of a representation of 
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some information such as a sentence, paragraph or story (Greeno, 1978). The 
representation constructed is based on the actual input, that is, on the 
specific materials presented to the subject and on the subject's conceptual 
knowledge (disibio, 1982). Conceptual knowledge includes grammatical 
knowledge, factual knowledge, and specific content-domain knowledge (Mayer, 
1982; Minsky, 1975). Judgments about the degree of a solver's jnderstanding 
are made in terms of the features of the representation. 

The def initial of problem representation that is used in this study is: 
A problem representation is a cognitive structure which is constructed by a 
solver when interpreting a problem on the basis of his domain-related 
knowledge and its organization. This definition is similar to that used by 
Chi, Feltovich, & Glaser (1981) in studying problem solving in physics and 
is consistent with the use of the term representation in the cognitive 
psychology literature where the term is widely used, particularly in the 
literature on the study of language comprehension. A discussion of the 
concept of representation follows. 

To study how the mind functions, cognitive psychologists present a 
subject with tasks which the subject interprets in terms of his or her 
conceptual knowledge. The result of this processing is called the mental or 
internal representation of the task (Mayer, 1978; Greeno, 1977) . The 
representation may or may not resemble the presentation iconically. For 
example, research has shown that the mental representation of the word 
"four" is likely to not be verbal, rather it is more likely to be "visual" 
(Shephard & Podgorny, 1978). Similarly, the symbol "4" is not likely to be 
represented mentally in terms of the shape of the symbol, rather in terms of 
an associated concept such as a pattern. Gn the other hand, a human face is 
Likely to be encoded visually, so the mental representation will iconically 
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resemble the perceived stimulus (Shephard & Podgorny, 1978). 

Sentences are typically process*. not as sequences of or even 

as sequences of words, but rather in terms of their meaning (h^ansford, 
1979). A string of ncnsense syllables, on the other hand, may be encoded 
exactly in that form, i.e. as a string of syllables or even as a sequence of 
letters, because it does not signify anything to the .ubject. Words, hence 
sentences, have meaning in as much as they point to and elicit associations 
with knowledge previously constructed by the subject. Persons studying 
language comprehension do not view the mental or internal representation of 
a sentence as consisting of the words in the sentence. Rather, the 
representation includes the meaning associated with the words and is 
necessarily linked with conceptual knowledge. Ebr example, the sentence 
"Ida borrowed the tablecloth from Jan" is linked mentally to the concept of 
"borrowed". Thus, included in the mental representation is the notion that 
the tablecloth was at one time in Jan's possession, possession then changed 
to Ida and possession has or will return to Jan. 

Since a mental representation ot a sentence includes links with prior 
conceptual knowledge, the representation is subject dependent. Consider the 
sentence "Bill bought the red car." An individual who does not know who 
Bill is or which car is being referred to as "the red car" will form a 
mental representation with relatively few associations. Imagine a friend of 
Bill's who was interested in buying the same red car. The mental 
representation that the friend has of the sentence will have relatively more 
associations than the representation in the previous case. Fc-r example, for 
the friend, the sentence will be linked, with the knowledge that he too 
wanted to buy the car and possibly to information about: the cost of the car. 
Previously constructed knowledge influences the mental representation 
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evoked by n stimulus. 

The view adopted here is that in mathematics problem solving, when an 
individual is presented with a problem, he or she uses the information to 
form a mental representation of the problem. As with sentence 
representation, problem representation includes more than is directly 
provided in the problem statement. Associations are established with 
conceptual knowledge. Different individuals have different conceptual 
knowledge and will make different associations with their knowledge. 
Relationships between elements also may be established depending cn the 
person's existing schemas. In some cases an individual may fail to mentally 
use some of the information provided by the problem statement, may establish 
some relationships contrary to the problem statement, or may fail to 
establish certain relationships. Consequently, problem representation is 
quite subject dependent. Further, as a subject attempts to solve a problem., 
new relationships between problem elements or between problem element • and 
other knowledge the subject already has, may be formed. Thus the formation 
of a problem representation is a dynamic constructive process and depends on 
the the individual forming the representation. 

Purpose 

The purpose of this study was to develop a list of characteristics of 
problem representation that are indicative of understanding. Greeno's view 
that the degree of understanding of a problem is indicated by the nature of 
the problem representation was adopted. 

The study was carried out in two phases. Phase cne had as its goal 
the development of a list of characteristics of problem representation 
indicative of understanding. Greeno's criteria of understanding in problem 
solving, based on the theory of language comprehension, (Greeno, 1977) and 
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the qualities of schema outline I by Skemp (1979) were useful guides in the 
development . 

Phase two consisted of determining the relationships between 
performance on a set of follow-up tasks and the ratings of characteri sties 
of problem representation outlined in phase cne . The question to be 
answered in phase two was: 

"Do problem solving .protocols which have higher ratings on the 
problem representation characteristics identified in phase one 
correspond to greater success on the follow-up tasks than problem 
solviag protocols which have lower ratings on the problem 
representation characteristics?" 

Each of the follow-up tasks was selected because of its relevance to 
understanding. It was assumed that successful performance on the follow-up 
tasks indicates that the original problem was solved with understanding. 

Phase One 

The purpose of the first phase was to develop a list of problem 
representation characteristics that are indicators of understanding. 
Method 

The subjects for phase cne were sixteen students, ages 16 to 24, 
including nine undergraduate and five graduate students a. a large 
midwestem university and two high school juniors. Subjects were selected 
who were able to convey, verbally .or through their written work, their 
mental processing while solving mathematics problems. Subjects were 
scheduled for two-hour problem solving interivews with the experimenter, in 
pairs when scheduling permitted, four of the subjects were interviewed 
individually. The idea of using pain; of subjects in collecting data on 
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problem solving is due to Schoenfeld (1981). His rationale is that when two 
subjects work together to solve a problem they must reveal what they are 
thinking to their partner. Consequently, more verbal izaticn occurs without 
prompting or interference from the interviewer. During the interview 
subjects worked together to solve a variety of mathematics problems while 
thi. ing aloud f One example of the problems used is the following: 

The surface of Clear Lake is 35 feet above the surface of Blue 
• lake. Clear Lake is twice as deep as Blue Lake. Tfrn bottom of 

Clear Lake is 12 feet above the bottom of Blue Lake. Sow deep are 

the two lakes? 

The problems were presented one at a time. The experimenter gave no 
feedback about the correctness of the solution but asked questions when 
necessary to encourage the subjects to reveal their thought processes. The 

i 

interviews were audio- tape recorded. The experimenter kept a detailed 
written record of observations during the problem solving interview. This 
written record aided in the coordination of a subject's verbal and written 
records and also served as a means of recording significant and interesting 
behaviors that were not apparent by later review of the subject's written 
work or the audio tape. The subject' r written work, the audio-tape, and the 
experimenter's written record formed the protocol. 

Subjects were asked to try to complete each problem before, proceeding 
to the next problem. However, subjects were notified after they-had worked 
on a problem for twenty minutes without completing it so that they would 
have ample opportunity to attempt a large number of the problems. The 
number of problems completed differed for different subject pairs. Since 
the purpose of these interviews was to gain information useful to the 
development of a list of characteristics of problem representation it was 
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not essential that all subjects complete exactly the same set of problems. 

. The protocol analysis in phase one of this study proceeded as follows. 
An initial list of anticipated observable events was made, ihis list 
included such things as attention to types of notation used, diagrams used, 
amount of rereading of portions of the problem statement and which portions 
are reread, evidence of planning, identification of problem components, and 
evidence of use of information not explicitly stated in the problem. The 
Ericsson and Simon model of verbal reporting suggests that periods of 
silence may be indicative of a reorganization of a representation or 
strategy (Ericsson & Simon, 1980). Consequently, observations of the above 
and similar events were made throughout the problem solving process with 
careful attention to changes after periods of silence. Schoenfeld (1981) 
had suggested that it is possible to identify decision points in a problem 
solving protocol where a solver may be redirecting his problem solving. 
According to Schoenfeld, problem solving protocols can be divided into 
"macroscopic chunks of consistent behavior", e.g. reading, analysis, 
exploration, transition, which he labels episodes. The points between 
episodes are called decision points. If decision points between episodes 

t 

are potential places where a solver redirects his problem solving, they may 
be points where the solver revises his problem representation. Although the 

protocols were not parsed into episodes in the study reported here, 

j 

attention was given to evidence of changes in the characteristics of the 
representation at apparent decision points. 

Protocols were analyzed using the initial list. Modifications were 
made in the list to account for important observations not accounted for by 
the initial list. The analysis and modification process continued until 
clarification emerged, i.e. until the list adequately accounted for observed 

1 0 



Problem Representation Characteristics 

12 

behaviors. The characteristics of problem representation that resulted are 
necessarily the experimenter's own interpretation of the evidence but are 
founded cn the experimenter's understanding and knowledge of related 
research. The analysis, was a continual process and was concluded when a 
consistent account emerged. Because it offered additional helpful 
information, data from a pilot study, conducted in a manner similar to phase 
cne but using individual interviews rather than pairs, were also used in the 
development of the list of characteristics of problem representation. 
Results 

' Biase one resulted in a list of characteristics of problem 
representation that can be used to evaluate a problem solving episode. 
Greeno (1977) identified correspondence, coherence, and connectedness as 
qualities of problem representation indicai * of understanding. This study 
extends Greer :./s work by providing a means of assessing these qualities and 
by elaborating additional representation characteristics. The 
characteristics developed here fall into three broad categories: content, 
external code, and processes. The content category is used to evaluate 
"what" is represented, the external code category to evaluate "how" the 
content is represented, and the processes category to evaluate specific 
features of how the problem solver proceeded in developing the attained 
problem representation. Each category is divided into subcategories tor 
purposes of providing a detailed characterization of problem representation. 
The complete list of characteristics developed is shown in the problem 
representation instrument in Figure 1. 
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Insert Figure 1 about here 



Ihe distinction between the content and the code of the representation 
parallels similar distinctions made in studying language representation 
(Glass, Holyoak, & Santa, 1979). The inclusion of the processes category 
characterizes the way the problem representation was formed. tC , 



In this research the position was taken that there is no one ideal 
representation for a problem. Consequently, each problem representation is 
evaluated on its own merits rather than by comparison with same 
predetermined representation. The method of evaluating the character istxcs 
identified reflects this position. FOr example, within the content 
category, the subcategories accuracy and completeness are evaluated by 
identifying inaccuracies ruri incompleteness in the solver's representation. 
The default is accurate and complete. 

The content category contains the subcategories accuracy, completeness, 
and general izability. Accuracy is a measure of the extent to which the 
solver's representation is consistent with the statement of the problem. A 
problem representation is considered to be accurate unless inaccuracies are 
found. Inaccuracies, or errors, in problem representation may be due to a 
variety of factors. Fhase one of the study resulted in the identification 
of the following factors as causes of inaccuracy: encoding error, 
unjustified assumption, incorrect inference, lack of knowledge, 
computational error, and inaccurate goal. Definitions of these factors and 
their ratings are given in the appendix. 
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Completeness of a problem representation refers to the extent to which 
the information extracted from the problem statement and the relationships 
established while building and elaborating the representation are sufficient 
for the solution of the problem. In the process of solving a problem the 
solver must form relationships by encoding the problem statement in terms of 
his or her conceptual knowledge. To be complete a problem representation 
must contain needed explicit and implicit relationships and a representation 
of the goal. A relationship is called explicit if it is based only on 
information provided explicitly in the problem statement. A relationship is 
called implicit if it is inferred from aspects' dfStlje problem statement. 
Incompleteness in problem representation can occur in several different 
ways. Phase one of this study identified the following factors as causes of 
incomplete problem representation: absence of needed explicit relationship, 
absence of needed implicit relationship, lack of knowledge, and absence of 
goal. The definitions of these factors and their ratings are given in the 
appendix. 

Another characteristic of the content of representation considered in 
this research is general izability. The general izability of a problem 
representation refers to the extent to which the representation is useful 
for solving problems similar in structure to the given problem.. A precise 
definition and the rating method are given in the appendix. 

The code of a representation refers to "how" the content is 
represented. In this study, there was no attempt to assess the internal 
code, only the external code. The solver's written work and verbalizations 
are taken as external code. Characteristics of external code identified by 
this study as important for describing the nature of the representation are 
level of abstraction, analogical versus analytical features, and specific 
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types of code, such as a diagram, equations, chart or list. A code is 
analogical if it ioonically resembles what is being represented. A cede is 
called analytical if it consists of arbitrary relationships between the 
representation and what is being represented. Language, verbal, written, 
and mathematical, are all examples of analytical code. Definitions of the 
e;cternal code categories and their ratings are given in the appendix. 

The third broad category of problem representation characteristics 
identified in fhase one describes processes by which the representation is 
established. The factors selected for this category are: identify versus 
build, immediacy of relationships, types of connections, and strength of 
connections. Identify versus build refers to the extent to which the 
problem solver approaches the problem by treating it as a type for which he 
has available a schema or general representation which indicates the 
solution process. Immediacy of relationships indicates the extent to which 
the solution process is dominated by the establishment of relationships or 
by carrying out needed mathematical procedures. Types of connections refers 
to the extent to which the connections established are based en rote 
memorization or syntactic processing versus conceptual or semantic 
processing. Strength of connections refers to the solver's confidence in an 
established representation as evidenced, in part, by persistence with the 
representation. Precise definitions of these factors and their ratings are 
given in the appendix. 

The evaluation of a problem representation en the characteristics 
identified in this study involves subjectivity. Specifically, knowledge 
about a subject's mathematical knowledge and background influences the' 
ratings. Further, to uc>e the problem representation instrument effectively 
the evaluator should be present during the problem solving interview. This 
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limitation is not viewed as. a weakness since typically intuitive evaluations 
of "understanding" require that the evaluator have considerable knowledge 
about the solver and his solution process. Further, the goal of this study 
was not to produce an instrument for use by independent evaluators, rather 
it was to explicate the nature of problem representation by identifying 
characteristics that are indicative of understanding. The characterization 
given hpre is best clarified through examples which illustrate the 
characteristics and their ratings, and which clarify the distinctions 
between the various characteristics. A detailed discussion is given in 
Yackel (1984). 



ftiase Two 

The purpose of phase two was to verify that the characteristics of 
problem representation identified in phase one are in fact indicative of the 
understanding attained by the solver. 

Method 

Riase two consisted of presenting subjects with problem solving tasks, 
assessing the problem representations developed during the problem solving 
in terms of the characteristics outlined in phase one, and then presenting 
subjects with follow-up tasks. The follow-up tasks were selected so that 
success on the tasks could be reasonably assimed to indicate understanding 
of the original problem task. Performance en each follow-up task was then 
compared to the ratings given on the problem representation characteristics. 

The subjects for this phase of the study were 36 students enrolled in 
an introductory level statistics course, taught by the experimenter, at a 
large midwestern university, who volunteered for the study. All of the four 
classes, freshman, sophomore, junior, and senior, were represented. There 
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were an equal number of males and females and the students represented a 
variety of ability levels. 

The follow-up tasks used were recollection of the problem immediately 
upon its completion, solution of a similar problem, .reation of a problem 
with a similar solution method, and recollection of the problem at the end 
of the interview. Use of the problem recollection tasks is based on work of 
Silver (1979) and Ikrutetskii (1976). Use of the similar problems task to 
assess understanding of a problem is based on work of Gagne (1966) and 
Greeno (1977). Use of the problem creation task as a means of assessing 
understanding is based on work of Krutetskii (1976) . 

Subjects were interviewed individually, by the experimenter in two-hour 
sessions. Subjects were asked to think aloud during the interviews which 
were audio- taped. In the interview each subject was presented with four 
problems to solve and four accompanying follow-up tasks. The problems used 
for the problem solving tasks were problems which require no mathematical 
knowledge beyond arithmetic for successful completion. The problems ware 
presented one at a time, typed on individual cards. A subject was allowed 
45 minutes to solve a problem. Upon couplet ion of the problem the 
experimenter removed the problem card and the solver's written work. The 
first three follow-up tasks were then presented. 

The first follow-up task was the immediate recollection of the problem. 
Subjects ware asked to "Repeat the problem statement." If a subject did not 
understand the task he was asked, "What did the problem on the card say?" 
The subject responded verbally to this task. 

Upon completion of the first follow-up task the subject was presented 
with a similar problem, typed on a card. The subject was not told that this 
was a follow-up task or that the problem was similar to the original 
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problem. Upon completion of the problem the experimenter removed the 
problem card and the solver's written work.. Follow-up task 3 wa« then 
presented. 

For follow-up task 3 subjects were instructed to "Make up a problem 

having a solution method like the solution method of , (the name of the 

problem solving task)." It should be noted that the problem solving task 
was not the problem the subject had just completed since the similar problem 
had been solved in the intervening time. Subjects did not have access to 
either the statement of the original problem, the statement of the similar 
problem, or their written work for either problem during this task. 



As a fourth follow-up task, all subjects, at the end of the interview, 



Since the subject had also completed the similar problems and had not been 
told that some tasks were follow-up tasks, it was necessary for the 
experimenter to identify for the subject which problems were to be recalled. 
This was done by saying something such as, "You did a problem about a 
football league and the draft. Tell me what the problem said." If the 
subject proceeded to explain his problem solution the experimenter said, 
"Tell me what the problem statement on the card was." 

Throughout the interview the experimenter kept extensive notes of the 
subject's activity. These notes were used to coordinate the subject's 
written work with the audio-tape as well as to record information that would 
not be apparent from later review of the solver's written work or the 
audio- tape. 

For each of the four initial problems presented in phase two, the 
subject's problem representation was characterized using the instrument 
developed in rhase one. The solver's written work, the audio-tape and the 




were again asked to restate each of the four problems previously presented. 
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experimenter's written record were used in this evaluation process. The 
follow-up tasks were evaluated as follows. The immediate and final problem 
recollections were rated as correct in details only, correct in structure 
only, correct in both details and structure or correct in neither. The 
solution of the similar problem was evaluated on appropriateness of solution 
method and correctness of answer. The problem created by the subject was 
rated as similar in details to the given problem, same in structure as the 
given problem, both or neither. 

The data obtained from phase two of the study were analyzed as follows. 
Each of the items listed en the problem representation instrument was 
treated as a separate random variable. For example, within the category 
accuracy there were six variables, encoding error , unjustified assunption , 
incorrect inference , lack of knowledge , anil computational error , and 
inaccurate goal. Within the category external code, analogical vs 
analytical was taken as one variable. The portion of the external code 
category labeled "types" was coded so that each type of code formed a 
separate variable. Presence of that type of code was rated 1 and absence 
rated 0. EU. example, the variable diagram was rated 1 if there was a 
diagram present and was rated 0 if no diagram was present. The variables 
defined by the problem representation instrument are referred to as the 
representation variables. There are 23 variables in all including answer on 
original problem . Even though it is not viewed as a characteristic of 
problem representation in this study, answer on original problem was 
included since it records the product of the problem solving process and as 
such the result of the solver's use of his or her problem representation. 
All of these random variables except analogical vs analytical axe ordinal 
level. An additional variable recorded for each of the problem solving 
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tasks was solution time in minutes. The follow-up task variables were 
immediate recollection , final recollection , problem creation , solution time 
en similar problem, method on similar problem, and answer on similar 
problem. Five of the six follow-up variables, the exception being solut ion 
time en similar problem, are ordinal level. 

Three different analyses of the data ware conducted. The first was the 
consideration o^ frequencies of the representation variables and the 
follow-up task variables except for solution time en similar problem. Those 
variables which ware heavily concentrated cn a single variable value were 
omitted from the second statistical analysis of the data. 

The second analysis conducted was measures of association between the 
representation variables and the follow-up task variables. Since almost all 
of these variables were ordinal level, the appropriate measure of 
association was the Kendall's tau correlation coefficient. When the pair of 
variables to be correlated had equal number of possible values Kendall's tau 
b was used, otherwise tau c was used, (Kendall, 1970). Kendall's tau b and 
tau c are appropriate when the data have a large number of ties as was the 
case here (Agresti & Agresti, 1979; Kendall, 1970). 

The final analysis was the computation of Kendall's tau correlation 
coefficients for the representation variables with each other. This 
analysis determines whether or not the variables are related to each other. 
Results 

The frequency data shows that each of the representation variables 
achieved each of its values except for certain variables in the accuracy and 
completeness categories. The variables encoding error , unjustified 
assumption , lack of knowledge and inaccurate goal , within the accuracy 
category, and the variables lack of knowledge and absence of goal, within 
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the completeness category, were concentrated cn the highest possible value. 
This means that for each of these variables all or most of the instances 
indicated no inaccuracy or incompleteness due to these factors. 

The follow-up task responses for immediate and final problem 
recollection were concentrated cn correct in both details and structure, 
with final recollection responses slightly less concentrated than the 
immediate recollection responses. The problem recollection follow-up tasks 
did not provide as much information as anticipated. The variables used to 
assess performance cn the similar problem task and the problem creation task 
proved more useful, especially method a»d answer cn - the similar problem 
task. For a discussion of the frequencies results see Yackel (1984). 

The major result of phase two of the study was that the characteristics 
of problem representation identi f ied in phase one of the study are 
indicative of understanding When measured by answer or method on the similar 
problem task and by the problem creation task. Table 1 shows? the 
correlations of the representation variables and the follow-up task 
variables. Strong positive correlations indicate that high ratings on the 
representation variables are associated with high ratings on the follow-up 
tasks and hence are indicative of understanding on the original task. 



Insert Table 1 about here 



Correlations of follow-up task variables with the immediate and final 
recollection tasks were relatively small in magnitude when they were 
significant. The concentration of the recollection response*: on a single 
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value limited the utility of these taste in differentiating between 
qualitatively different problem representations, there are several ' reasons 
for this. These are discussed in detail in Yackel (1984). 

Kendall's correlation coefficients were computed to determine the 
extent of relationship between the representation variables. Table 1 shows 
that the categories of variables on the problem representation instrument 
are not independent of each other but that within the category accuracy the 
variables are, for the most part, unrelated and within the category 
completeness some of the variables are not related. Even when a significant 
correlation exists between variables it is not appropriate to conclude that 
the variables are not measuring distinct characteristics. For example, 
absence of needed explicit relationship and absence of needed implicit 

relationship certainly measure two distinct characteristics of problem 

i 

representation, yet they are positively correlated. Bpr the problems and 
subjects used in this study high ratings on one variable occurred 
simultaneously with high ratings on the other variable and low low 
combinations also occurred. These occurrences outweighed any high low 
combinations which occurred. 



Discussion 

The overall plan of the study was to identify characteristics of 
problem representation potentially indicative of understanding, in phase 
cne, and to verify that the characteristics are in fact indicative of 
understanding, in phase two, through the use of follow-up tasks assumed to 
be indicators of understanding. The study has shown that characteristics of 
problem representation which are indicative of understanding are accuracy, 
completeness, general izability, and certain process variables. Specific 
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causes of inaccuracy and incompleteness have been identified. Those which 
were not concentrated cn a single • triable value wore shown to be indicative 
of understanding, as assessed by most of the follow-up tasks. Consequently 
the meaning of "under standing' 1 in mathematics problem solving is clarified 
through the characterization of problem repres jncaticn developed here. 

The degree of understanding ejfoibited by ? solver in a mathematics 
problem solving task can be assessed directly by coniideriftg characteristics 
of the problem representation formed by thi solver. It is not necessary to 
use subsequent tasks to assess the degree of understanding of the original 
task. 

In this study incorrect inference and inaccurate goal were the most 
frequently occurring causes of an inaccurate problem representation. 
Absence of needed explicit relationship and absence of needed implicit 
relationship were the most frequently occurring causes of an incomplete 
problem representation. Several of the factors listed as causes of 
inaccuracy and incorapeleteness were observed only infrequently in this 
study. The infrequent occurrence of sane of these is e^lained by the very 
specific nature of these factors as causes of inaccuracy or incompleteness 
and by the criteria for problem selection used in this 3tudy. Seme of the 
variables, such as encoding error , are very specific tut are necessary to 
provide a complete description of sources of inaccuracy and incompleteness. 
A variable such as incorrect inference is less specific and hence 
encompasses wore errors. Consequently its frequency as a aause of error is 
much higher. The criteria for problem selection used in this study limited 
the likelihood of occurrence of some of the factors as causes or error or 
iraccuracy. For example, problems were selected which require no 
mathematics knowledge beyond that of the typical college student, thus 
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reducing the f requency of lack or ^knowledge as a cause of error or of 

• .... 

incompleteness. 

Analysis of the problem solving protocols and performance on the 
follow-up tasks showed that a diagram plays at least two significant, tut 
distinct roles in problem solving. It serves as a means of expressing 
information in the solver's current mental representation, that is, a solver 
uses a diagram to record spatial information given in the problem statement 
or information he or she has derived from the problem statement. Once drawn 
it also serves as a means of aiding the solver in further developing the 
representation, especially in establishing additional relationships that 
have spatial features. This second function is especially important since 
in problem solving a major task is to establish relationships between 
problem components. 

Also of interest is the role diagrams serve in recalling problems. In 
this stoiy seme subjects, when given the problem creation task which 
required creation of a problem with a similar solution method to the' 
original problem, recreated a diagram drawn for the original problem as an 
aid in its recollection, thus providing evidence that their internal code 
was spatial in nature. Furt 1 r investigation of the role of diagrams in 
problem soling and of the potential of the problem creation task in 
assessing spatial features of the internal code of a prbblem representation 
is indicated. 

Implications 

This study has several implications for research. First it lias shown 
that a problem solving task can be meaningfully assessed for degree of 
understanding. This can be done by considering the characteristics of 
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problem representation identified in this study. It is not necessary to use - 

4 

a subsequent task to assess the degree of understanding exhibited during the 
problem solving episode * The construct of mental representation .is useful 
in studying problem solving. The term problem representation then assumes a 
much broader meaning than it has been given in most previous mathematics 
education contexts. Terminology used by cognitive psychologists, such as 
"code", and "analog" and "analytic" code, is useful in describing problem 
representations . 

Second, this study has shown that it is useful to study problem solving 
from a global approach. Much has been learned from problem solving. process 
research which studies a problem solver's activity by checking processes 
used and recording the sequence of their use. Such research looks at 
problem solving from a microscopic view. Schoenfeld (1981) has called for 
research which takes a macroscopic view of problem solving. Use of the 
construct, problem representation permits a macroscopic view; investigating 
the quality of a problem representation using the problem instrument 
requires analysis of the problem solving protocol as a whole. Further 
research can take advantage of the development and clarification provided 
for problem representation and understanding in problan solving presented in 
this study. 

The research reported in this study has important implications for the 
teaching of problem solving. Current emphasis in problem solving is on the 
teaching and use of heuristics. Such emphasis is well founded. Familiarity 
with and ability to use heuristics in problem solving is essential for the 
successful solver. This research shows that attention to the formation of a 
problem representation is another important aspect. Specific aspects of 
representation such as accuracy and completeness need to be emphasized by 
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teachers. Students can be specifically directed to attend to the Qualities 
of accuracy and completeness. The intent is not that a student will be able 
to judge his representation for accuracy and completeness. Bat, that he 
will be explicitly aware that failure to solve a problem may be due to 
interpreting the problem differently than intended or failing to use 
additional information or establish relationships. The distinction between 
explicit and implicit information can be made. Once the distinctions are 
clarified students can become cognizant of information used in problem 
solving which is explicit and information which is implicit. General 
awareness on the pirt of a student that implicit information is often needed 
to solve a problem should encourage him to consider potentially related 
information. 

The various factors causing inaccuracy and incompleteness that have 
been outlined can be useful to teachers in identifying inadequacies in a 
student's problem representation. These factors can be used by the teacher 

4 

to help provide direction to a student without telling the student 
specifically what is in error or lacking or exactly how to proceed to 
successfully solve the problem, successful use of specific aspects of 
representation, such as accuracy and completeness, by teachers presumes that 
V^ey have extensive experiences solving problems themselves and observing 
others solve problems. J 

Use of the problem creation tlask has shown its potential as a teaching 
tool . Students should be asked to create problems wi th similar solution 
methods to problems they have solved. In the process of problem creation, 
students are forced to consider the relationship between various 
mathematical relationships and the verbal statements which express those 
relationships. Problem creation is a difficult task for students. Success 
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on this task may require a higher degree of understanding than successful 
solving of a similar problem. 

Two additional implications for teaching problem solving relate to the 
identify vs build and the immediacy of relationships variables . Students 
must learn that in problem solving (as defined herein) an appropriate 
problem representation will not be available within the solver's existing 
knowledge. Problem representations are built by establishing relationships 
between problem components and possible additional knowledge the solver 
identifies as relevant to the solution. Establishing these relationships 
typically dominates the problem solving process. Sarrying out necessary 
mathematical procedures and computations is secondary. Students should not 
e.Npect to know exacbly what to do immediately. To use Polya's terminology, 
"carrying out the plan" is secondary to "understanding the problem" and 
"devising a plan". 

There are several possible ways to help students learn the distinction 
between establishing relationships and carrying cut mathematical procedures. 
One way is to have students outline possible solution plans rather than to 
try to solve the problem. Schoenfeld (1982) has used this approach although 
for a different purpose. A simple way to emphasize carrying out 
mathematical procedures is to provide students with aids that minimize the 
amount of time and effort required to complete the procedures, such as 
calculators, integration tables, and lists of formulas. Finally, students 
are heavily influenced by the way they are evaluated. Tests that emphasize 
problem formulation through the types of questions asked and the way they 
are scored have potential for helping students make the distinctions 
suggested here. 

A final implication of this research for teaching problem solving is 
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that students need to have the capability to express their mental 
representation, in most cases students are unable to completely solve 
problems mentally without using an external expression of their thoughts, 
such as paper and pencil work. Students need to be able to use diagrams and 
mathematical lc^iguage, such as conventional symbolism and notation to 
effectively express their mental representations . Ability to use 
mathematical language and diagrams facilitates students' development of a 
problem representation. In this study tie experimenter observed that most 
subjects read a problem and either durinc the first reading or immediately 
thereafter recorded, on paper, informaticn\they obtained from the verbal 
problem statement. Repeatedly it was observed that when a subject did not 
know how to express on paper what he had just read, he would stop as if no 
progress could be made with the problem solving until that information could 
be expressed. In many of these cases it was difficult for the subject to 
come to the decision to proceed without having expressed cn paper the 
information provided by a sentence or a phrase. Without directly 
verbalizing it, most of these subjects seemed to be implying that unless 
they could express the meaning of the phrase or sentence cn paper, they had 
not established it mentally, apparently, ability to use mathematical 
language and notation facilitates the development of a problem 
representation. A confounding factor in this study was that roost of the 
subjects had little previous experience solving problems. It is quite 
likely that many of them intended to solve the problems by manipulating 
symbols and were stymied if they could not find an appropriate symbolic 
expression for the problem. This further emphasizes the need for the 
distinction between the content of a problem representation and the external 
code used to express the representation. 
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Appendix 

Definitions of Characteristics of 
Problem Representation and their Ratings 

Accuracy of representation 
The following factors in this category will be rated 0,1,2. 

0 More than one error due to this cause. 

1 One error due to this cause. 

2 No errors due to this cause. 

Encoding error ; An error in encoding information occurs when the 
solver (s) misreads or misinterprets a word or phrase. There must be 
evidence from the protocol that had the encoding error been noticed a 
correct representation would have resulted. Tnis error is characterized 
by quick processing and failure to carefully analyze the problem 
information . 

Unjustified assumption : An error due to the solver (s) making an 
assumption that is not justified en the basis of the information 
provided in the problem statement. 

Incorrect inference ; An error made vfoen the solver (s) makes an 
erroneous inference from the situation described in the problem 
statement . 

Lack of knowledge : An error in the representation vAiich results from 
the solver (s) lacking seme knowledge, eg. in interpreting the meaning of 
the vords in the problem statement, or in translating the information 
into mathematical notation. 

Computational error t An error in the representation vhich results from 
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an error in computation made by the solver(s). 
The following factor will be rated 0,1,2. 

0 Inaccurate goal. 

1 The solver shows evidence of having identified the 
correct goal early in the problem solving but stops 
working after attaining a major subgoal. 

2 Accurate goal. 

Inaccurate goal : Tbe solver (s) has identified a goal different from the 
intended goal of the problem. 



Completener of Representation 
The following factors in this category will be rated 0,1,2. 

0 More than one emission due to this cause. 

1 One emission due to this cause. 

2 No incompleteness due to this cause. 

Absence of needed explicit relationship t A relationship explicitly 
given in tlie problem information is omitted. 

Absence of needed inplicit relationship ; An implicit relationship 
necessary for the solution is omitted. 

Lack of Knowledge ; Failure to encode, to assign meaning to, one or more 
portions of the problem statement. 
The following factor will be rated 0,1. 

0 Omission of goal. 

1 Goal present. 

Absence of goal ; The solver is unable to determine a goal for the 
problem, i.e. has no specific goal. 
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General izability of representation (0,1,2) 
The generalizability of a problem representation refers to the extent to 
which the representation is useful for solving similar problems. 

0 (isolated): The representation is useful only for the problem given. 
Ncnsystematic trial and error falls at this level. 

1 (seme evidence of integration): Sane aspects of the representation 
will be useful for solving similar problems. Systematic trial and error 
as well as general statements made verbally but not explicitly written 
down will be taken as evidence of some integration. 

2 (integrated): The problem representation is descriptive of or uses 
notation or a solution method which clearly indicates that it is one of 
a class of similar problems. A problem involving similar relationships 
could be solved using the same representation making needed 
modifications to account for the new data. 



External Cede 

Analogical vs analytical . The following types will be checked if they 
are observed to occur. 

none: Essentially no external code is used. Several numbers nay be 
written on the solver's paper but nothing else. 

analogical only: The code used in sane ways resembles what is being 
represented, i.e. it has seme properties similar to actual perception or 
has seme features of what is being represented. 

analytical enly: The code is based en an arbitrary relationship between 
the representation and what is being represented, eg. mathematical 
symbolism or notation. 

both analogical and analytical: Both types of code' are present in the 
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solver's work. 

Level of abstraction (0,1,2) 

Level of abstraction refers to the extent to which what is expressed by 
the code is more general than the input which is actually presented, 
when more than one code is present the roost abstract code will be rated. 

0 (low): The external code used describes only what is actually 
presented. There is no generalized notation. 

1 (moderate): Limited use is made of symbolic notation, equations or 
diagrams which abstract the mathematical features of the problem. 

2 (high): Extensive use is made of symbolic notation, equations, 
diagrams which abstract the mathematical features of the problem. The 
code expresses a general mathematical model. 

Types of code : 

To facilitate the above rating of level of abstraction the following 
types of code will be checked when they are observed to occur. The 
category "other" provides for indicating a form of code not among those 
listed that is observed to occur and is viewed as relevant to the 
determination of the level of abstraction, 
diagram 

symbolic notation 
equations 
chart or list 
other 

Process Of Establishing The Problem Representation 
The variables in this category characterize the process of establishing the 
problem representation . 
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Identify vs build (0,1,2) 

0 (identify): The solver treats the problem as one of a type for which 
he already knows a solution procedure and identifies a representation 
that can be used to solve the problem. 

1 (mainly identify, some evidence of building): The solver identifies 
the problem as one of a type he knows how to solve but mast establish a 
number of relationships and processes semantically for that purpose. 

2 (build): The solver establishes relationships on the basis of the 
information in the pi obi em statement and not on the basis of treating it 
as a problem type. 

Immediacy of relationships (0,1,2) 

0 (low): The solver (s) is slow to sense the relationships. An 
appropriate representation is not established until near the end of the 
solution process or not at all. 

1 (moderate): Establishing the necessary relationships dominates the 
solution process. Some time may be spent initially on exploration. 
Relationships my be modified during the solution. Some time is spent 
carrying out the necessary mathematical procedures but this is 
secondary. 

2 (high): The needed relationships are established almost immediately 
by the solvers . Carrying out the necessary mathematical procedures 
dominates the solution process. Establishing the needed relationships 
is secondary. 

Types of connections (0,1,2) 

0 (associative): Connections are established on the basis of rote 
memory or statement syntax. 

1 (mainly associative, some evidence of conceptual): Most of the 
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ccnnecticns are associative. There is evidence that sane are 
conceptual . 

2 (conceptual): There is evidence that the connections are conceptual 
or that most of the processing is semantic. 
Strength of connections (0,1,2) 

Strength of ccnnecticns refers to confidence in the problem 
representation. 

0 (weak): Solvers have not attained a solution or have no confid'sioe in 
their representation. Relationships established are readily abandoned 
at the suggestion of a solution partner or an observer. 

1 (moderate): Connections are used in a very tentative way. The solver 
expresses seme uncertainty about some of the relationships that have 
been established. Obtaining an answer to the problem which the solver 
has reason to believe is correct may be the only way the solver is 
certain that the relationships are correct. 

2 (strong): The solver exhibits strong confidence in the problem 
representation . 

Answer on Original Problem (0,1) • 

0 (incorrect): No answer is given or the answer given is not correct. 

1 (correct): The answer given to the problem is .correct. 
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Table 1 l< 
Correlation Coefficients of Representation Variables 
with Follow-Up Task Variables 



recollection 
immediate final 



Follow-up Task Variable 

• similar problem- 
method . answer time 



problem 
creation 



Representation variable 
Accuracy V 

incorrect inference 

inaccurate goal 

accuracy total 
Completeness 

absence of heeded 
explicit relationship 

absence of needed 
implicit relationship 

completeness total 
Gene ral i zabi li ty 
Level of abstraction 

of external code 
Processes 

identify vs" build 

immediacy of 
relat ionships 

types of connections 

strength of connections 

process total 
Answer on Original Problem 



,12** 

,03 



13* 
12** 



11 p<.05 
** p<.0.1 



40** 

07 




\ 



.27** , 



.13** 


.20** 


.43** 


.44** 


.13* 


.30** 


.10** 


.12** 


.17** 


.15** 


-.11 ) 


.21**\ 


.04 

.18* 

.04 


,05 
.11* i 
.13* 


.62** 
.61** 
.57** 

* 


.42** 
.43** 
.45** 


.22** 

.10 

.19** 


.25** 
.29** 
. .16* 


.06 


.07 


.34** 


.14* 


.12 


•J. 02 

t 


.02 


-.05 


.23** 


.24** . 


.41** 


/ 

.16** 


.03 
.05 
.07 
.07 
.07 


.08 

.07 

.20** 

.11* 

.16* 


.61** 
.32** 
.36** 
.61** 
.44** 


.36** 

.34** 

.20* 

.46** 

.39** 


.3.8** 
.22** 
-.3 9** 
.16** 
' .21** 


.24** 
.26** 

.:>i** 

.34** 
.27** 
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PROBLEM REPRESENTATION INSTRUMENT 



CHARACTERISTICS OF PROBLEM REPRESENTATION 



Subject^ 
Problem 
Date 



A. Accuracy of representation 

factors determining inaccuracy 
of representation 

encoding error 

unju p cified assumption 

incorrect inference 

lack of knowledge 

computational error 

inaccurate goal 

8. Completeness of representation 

factors determining incompleteness 

of reDre* f»nt* tion 

absen.e o« needed explicit relationship 
abser. e n- needed inr.olici t relationship 
lack r knowledge 

absence of goal 



C. General l2abil1ty of representation 



more than one one error no errors 



error 
0 
0 
0 
0 
0 

0 

yes 



more than one 
omission 
0 
0 
0 

0 

yes 



2 
2 
2 
2 
2 

2 

no 



one omission no omissions 



1 
1 
1 

1 

no 



2 
2 
2 



0 

isolated 



1 

some evidence of 
Integration 



Integrated 



External code 

Analogical vs analytical 
none_ 

analogical only 

analytical only~ 

both analogical and analytical 



Level of abstraction 

0 1 2 

low moderate high 

types: 
d'anra 

symbolic "notation _ 

equations _ 

or TlVf _ 
oth#r (deicrlMl 



42 

BEST COPY AVAlLABLb 



Problem Representation Characteristics 

41 



Process of establishing the problem representation 
identify vs build 

0 1 
identify mainly identify, some 
evidence of building 

immediacy of relationships 



0 

low 

types of connections 



1 

moderate 



2 

build 



2 

high 



0 1 
associative mainly associative 
some evidence of 
conceptual 



strength of connections 
0 

weak 



1 

moderate 



conceptual 



2 

strong 



Correctness of answer 



incorrect 



1 

correct 



Figure 1 
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